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Abstract — In this paper, we investigate in a unified way the 
structural properties of solutions to inverse problems. These 
solutions are regularized by the generic class of semi-norms 
defined as a decomposable norm composed with a linear operator, 
the so-called analysis type decomposable prior. This encompasses 
several well-known analysis-type regularizations such as the 
discrete total variation (in any dimension), analysis group-Lasso 
or the nuclear norm. Our main results establish sufficient 
conditions under which uniqueness and stability to a bounded 
noise of the regularized solution are guaranteed. Along the way, 
we also provide a strong sufficient uniqueness result that is of 
independent interest and goes beyond the case of decomposable 



I. Introduction 



A. Problem statement 
Suppose we observe 



y — ^xo + w, where ||ii;||2 ^ e 



where <i> is a linear operator from 



fN 



to 



y>AI 



that may 



have a non-trivial kernel. We want to robustly recover an 
approximation of xq by solving the optimization problem 



where 



X* e Argmin ^\\y ~ t&xlj + \R{x) , 



R{x) := \Vx\a 



(1) 



with L : M^ ^ R^ a linear operator, and || • |U : M^ -^ R+ 
is a decomposable norm in the sense of |1|. Decomposable 
regularizers are intended to promote solutions conforming to 
some notion of simplicity/low complexity that complies with 
that of uo = L*xq. This motivates the following definition of 
these norms. Throughout the paper, given a subspace V C R^, 
we will use the shorthand notation Ly = LVv, Ly = VyL*, 
and av = Vyct for any vector a G R^, where Vv (resp. 
Pyj-) is the orthogonal projector on V (resp. on its orthogonal 
complement V-^). 

Definition 1. A norm || • m is decomposable at u ^ M.^ if: 

(i) there is a subspace T C R^ and a vector e G T such 
that 



d\\ ■ Uu) ^{ae 



ar — e and \\ctT-^\\A ^ l} 



(ii) and for any z € T-^, \\z\\a = sup„gyi ||„||^^i(u, z) 
where \\ ■ ||^ is the dual norm of \ ■ \j^. 



From this definition, it can be easily proved, using Fenchel 
identity, that u E T whenever || • \\j\ is decomposable at u. 
Popular examples covered by decomposable regularizers are 
the £i-norm, the £±-£2 group sparsity norm, and the nuclear 
norm [|T|. 

B. Contributions and relation to prior work 

In this paper, we give a strong sufficient condition under 
which ([Til admits a unique minimizer From this, sufficient 
uniqueness conditions are derived. Then we develop results 
guaranteeing a stable approximation of xq from the noisy 
measurements y by solving ([T]i, with an ^2 -error that comes 
within a factor of the noise level e. This goes beyond 1 1 1 who 
considered identifiability under a generalized irrepresentable 
condition in the noiseless case with L — Id. ^2-stability for a 
class of decomposable priors closely related to Definition [T] is 
also studied in JS] for L = Id and general sufficiently smooth 
data fidelity. Their stability results require however stronger 
assumptions than ours (typically a restricted strong convexity 
which becomes a type of restricted eigenvalue property for 
linear regression with quadratic data fidelity). The authors 
in [3] provide sharp estimates of the number of generic mea- 
surements required for exact and £2-stable recovery of models 
from random partial information by solving a constrained form 
of ([T]i regularized by atomic norms. This is however restricted 
to the compressed sensing scenario. Our results generalize the 
stability guarantee of [7| established when the decomposable 
norm is £1 and L* is the analysis operator of a frame. A 
stability result for general sub linear functions R is given in Q. 
The stability is however measured in terms of R, and £2- 
stability can only be obtained if R is coercive, i.e., L* is 
injective. 

At this stage, we would like to point out that although 
we carry out our analysis on the penalized form dl}, our 
results remain valid for the data fidelity constrained version 
but obviously with different constants in the bounds. We omit 
these results for obvious space limitations. 

II. Uniqueness 

A. Main assumptions 

We first note that traditional coercivity and convexity argu- 
ments allow to show that the set of (global) minimizers of ([T]l 
is a non-empty compact set if, and only if, ker(<l>)nker(L*) = 
{0}. 



The following assumptions will play a pivotal role in our 
analysis. 

Assumption (SC^r) There exist ry e M^-^ and a e d\\- |U(i*a;) 
such that the following so-called source (or range) condition 
is verified: 

$*?7 ^ Lae dR{x) . 

Assumption (INJy) For a subspace T C M^, $ is injective 
on ker(L^j^). 

It is immediate to see that since ker(L*) C ker(L^j^), 
(INJt) implies that the set of minimizers is indeed non-empty 
and compact. 

B. Strong Null Space Property 

We shall now give a novel strong sufficient uniqueness con- 
dition under which problem ([TJ admits exactly one minimizer. 

Theorem 1. For a minimizer x* of ((TJ, let T and e be the 
subspace and vector in Definition\l\associated to u* = L*x*, 
and denote S = T"*". x* is the unique minimizer of ([U if 

{L*Th, e) < \\L*sh\\% \Jh e ker($) \ {0} . 

The above condition is a strong generalization of the Null 
Space Property well known in £i regularization 1^. 

C. Sufficient uniqueness conditions 

1) General case: A direct consequence of the above theo- 
rem is the following corollary. 

Corollary 1. For a minimizer x* of ([TJ, let T and e be the 
subspace and vector in Definition\l\associated to u* — L*x*, 
and denote S = T^. Assume that (SC^;*) is verified with 
\as\*A < 1. f^nd that (INJr) holds. Then, x* is the unique 
minimizer of ([U. 

In fact, it turns out that the above two results are proved 
without requiring some restrictive implications of Defini- 
tion [Ilii) of decomposable norms, and are therefore valid for 
a much larger class of regularizations. This can be clearly 
checked in the arguments used in the proofs. 

2) Separable case: 

Definition 2. The decomposable norm \\ ■ \\a is separable on 
the subspace T^ = S^V®W<zR^ if for any u e R^, 

I|wt-l|U = IkvlU + II^vkIU- 

Separability as just defined is fulfilled for several decom- 
posable norms such as the ^i or ^i — ip norms, 1 ^ p < +oo. 

The non-saturation condition on the dual certificate required 
in Corollary [1] can be weakened to hold only on a subspace 

V G S and the conclusions of the corollary remain valid, and 
assuming a stronger restricted injectivity assumption. We have 
the following corollary. 

Corollary 2. Assume that \ ■ \a i^ olso separable, with S = 

V © W, such that (SCx*) is verified with ||ay||^ < 1, and 
(INJy) holds. Then, x* is the unique minimizer of ([T]l. 



III. Stability to noise 



A. Main result 



1) General case: We are now ready to state our main 
stability results. 

Theorem 2. Let Tq and eo be the subspace and vector in 
DefinitionU] associated to Uq = L*xq, and denote Sq = Tq . 
Assume that (SCa;^) is verified with \\aso\\A "^ ^' '^^'^ ^^^^ 
(INJt;,) holds. Then, choosing X = ce, c > 0, the following 
holds for any minimizer x* of ([T]l 

||a;*-a;o||2 < Ce , 

where C = Ci (2 + c||77||2) + C24±^!^^^, and Ci > and 
C2 > are constants independent of rj and a. 

Remark 1 (Separable case). When the decomposable norm 
is also separable (see Corollary |2l), the stability result of 
Theorem |2] remains true assuming that ||Q;y||^ < 1 for 
V C 5*0. This however comes at the price of the stronger 
restricted injectivity assumption (INJy). To show this, the only 
thing to modify is the statement and the proof of Lemma |2] 
which can be done easily using similar arguments to those in 
the proof of Corollary |2] 

2) Case of frames: Suppose that L* is the analysis operator 
of a frame (ker(L*) = {0}) with lower bound a > 0, let L be 
a dual frame. The following stability bound can be obtained 
whose proof is omitted for space limitations. 

Proposition 1. Let Tq and cq be the subspace and vector in 
DefinitionU] associated to Uq — L*xq, and denote Sq = To . 
Assume that (SCa;^) is verified with ||aso||^ < 1, and that $ 
is injective on Ini(LTo). Then, choosing X — ce, c > 0, the 
following holds for any minimizer x* of ([T]l 



a^olb 



€C'e 



Ci(2 + cNb) + aii±«;.WC, >0 



where C 

and 6*2 > are constants independent of rj and a. 



Since ker(LJ ) C Im(LTo), the required restricted injectiv- 
ity assumption is more stringent than (INJt;,)- On the positive 
side, the constant C2 is in general better than C2. More 
precisely, the constant Cl, see the proof of Theorem |2] is 
replaced with ^/a. Note also that coercivity of R in this case 
allows to derive a bound similar to ours from the results in B . 
His restricted injectivity assumption is however different and 
our constants are sharper 

B. Generalized irrepresentable condition 

In the following corollary, we provide a stronger sufficient 
stability condition that can be viewed as a generalization of 
the irrepresentable condition introduced in [5 1 when R is the 
li norm. It allows to construct dual vectors -q and a which 
obey the source condition and are computable, which in turn 
yield explicit constants in the bound. 



Definition 3. Let T C M^ and e e M^, and denote S = T^. 
Suppose that (INJ^) is verified. Define for any u G ker(L5) 
and z G M*-'^ such that $*z G \Ti\(Ls) 



IC„,,(T,e) 



\Ve + us^{LsY^*z\\ 



-where 



^■. hi-^^h 



(Ls)+($*$S-Id)LT 
argmin i||$a;||^ - (/i, x) 

a:Gkcr(L o) 



and M^ is the Moore-Penrose pseudoinverse of M. Let u, z 
and u defined as 



and 



{u,z) = argmin IC„,^(T, e) 

uekGr(Ls),{z I $*zGlm(Ls)} 

u = argmin lCu,o{T,e) . 

Mgkcr(Ls) 



Obviously, we have 

ICa,.-(r,e) «; IC„,o(r,e) s; ICo,o(T,e) . 

The convex programs defining ICu^z{T,e) and lCufi{T,e) 
can be solved using primal-dual proximal splitting algorithms 
whenever the proximity operator of || • \\j\ can be easily 
computed ^. The criterion IC„^o(T, e) specializes to the one 
developed in [TOl when || • ||_4 is the £i norm. lGo,o{T, e) is a 
generalization of the coefficient involved in the irrepresentable 
condition introduced in |5| when R is the £i norm, and to the 
one in y.] for decomposable priors with L — Id. 

Corollary 3. Assume that (INJtq) is verified and 

ICu,z(2o,eo) < 1. Then, taking rj = ^'BLToeo + z, 
one can construct a such that (SCa^g) is satisfied and 
jasoW^ < 1- Moreover, the conclusion of Theorem^ remains 
true substituting 1 — ICa_z(To, Cq) for 1 — ||asoll^- 

IV. Proofs 

A. Proof of Theorem [7] 

A key observation is that by strong (hence strict) convexity 
of /U H->- IIj/ — /i|||, all minimizers of ([T]i share the same image 
under $. Therefore any minimizer of dl} takes the form x* + h 
where h G ker($). Furthermore, it can be shown by arguments 
from convex analysis that any proper convex function R has 
a unique minimizer x* (if any) over a convex set C if its 
directional derivative satisfies 

R'{x*]x- X*) > Q, xeC^x^x* . 

Applying this to ([T]i with C ~ x* + ker($), and using the fact 
that the directional derivative is the support function of the 
subdifferential, we get that x* is the unique minimizer of ([U 

if V /i G kcr($) \ {0} 

Q<R'{x*;h) = sup {v,h) 

vedR{x*) 

= sup {a, L*h) 

aea|MU(L-an*) 

= (e, L^h) + sup {as, L^h) 
= (e, L*^h) + \\L*shU ■ 



We conclude using symmetry of the norm and the fact that 
ker($) is a subspace. ■ 

B. Proof of Corollary [7] 

The source condition (SC^;.) implies that V h G ker($)\{0} 

{h, La) = {h, $*77) = ($/i, 7]) = . 
Moreover 

{h, La) = {L*h, a) ={L*rph, e) + {L*gh, as) . 
Thus, applying the dual-norm inequality we get 

(L^/i, e) ^ \\L*sh\U\\asrA < l^s^lU , 

where the last inequality is strict since Lgh does not vanish 
owing to (INJt), and ||as||^ < 1. ■ 

C. Proof of Corollary \2\ 

We follow the same lines as the proof of Corollary [T] and 

get 

{L*h, a) — {L*rph, e) + {Lyh, ay) + {Llyh, aw) ■ 
We therefore obtain 

{L^h, e) < ||Ly^lUI|ay||^ + ll-^w'^IUII"w'll!4 
<\\L*yh\U + \\L*wh\\A = \\L*shr^, 

where we used that h ^ ker(Ly), ||ay||^ < 1, separability 
and ||aw||^ =^ \\(^v\\a + II"m/||^ = l|as||^ < 1- ■ 

D. Proof of Theorem \2\ 

We first define the Bregman distance/divergence. 

Definition 4. Let D^{x,xq) be the Bregman distance associ- 
ated to R with respect to s G dR{xo), 

Df{x, xq) — R{x) — R{xa) — (s, x — xq) ■ 

Define D^{u, uq) as the Bregman distance associated to \\ ■ \\_a 
with respect to a £ d\\ ■ ^(uo)- 

Observe that by convexity, the Bregman distance is non- 
negative. 

Preparatory lemmata We first need the following key 
lemmata. 

Lemma 1 (Prediction error and Bregman distance convergence 
rates). Suppose that (SC^jp) is satisfied. Then, for any mini- 
mizer X* of ([TJ, and with X = ce for c > 0, we have 



D^,Jx*,xo)^D^{L*x\L*xo) ^ e 



(l + cN|2/2)^ 



||$x*-$a;o||2 ^ e(2 + CII77II2) . 

The proof follows the same lines as that for 

any sublinear regularizer, see e.g. |J9], where we 

additionally use the source condition (SC^:;,) and 
D^,^{x,xo) = Df^{x,x^) = D^{L*x,L*xo). 

Now since || • jj^ is a norm, it is coercive, and thus 
3 C^ > s.t. Va; G K^, ||a;|U ^ Ca\\x\\2- 



We get the following inequality. 

Lemma 2 (From Bregman to I2 bound). Suppose that {SCxa) 
holds with ||aso||^ < 1. Then, 



l|iSo(^'^-^o)lh 



< 



D^iL*x\L*xo) 

c.a(i-ii«soIi:,) 



Proof: Decomposability of || • \\a implies that 3v E Sq 

such that ||w||^ < 1 and \\L*g^{x* -xq)\\a = {L*g^{x* - xq) , v). 
Moreover, v + cq e d\\ ■ \\AiL*xo)- Thus 

D^{L*x\L*xo) ^ D^{L*x\L*xo) 

-D-^+eoiL*X*,L*Xo) 

= {v + Co — a, L*{x* — xo)) 
= {v - asa, L*g^{x* - xo)) 
= \\L*s„{x* - xo)\\a 

-("So, L*s^ix* -xo)) 
^ \\L*s„{x*~xo)\\A{l-\\asorA) 
^ CA\\L*s,{x*-xo)h{l-\\aso\\*A)- 



Proof of the main result 

\\x*-Xo\\2 ^ \\n,,^L',j{x* - Xo)\\2 

^ C$"i||$7'ker(Ljj(a;''-a;o)||2 

^ C^-^MX* - Xo)\\2 

+ (1 + C$-i||$||2,2)||^im(Ljj(a;* - a;o)||2 , 

where we used assumption (INJ^o), i-e., 

3C<t,>0 s.t. ||$a;||2 ^ C$||x||2, Vx £ ker(LJJ . 

Since L*g is injective on the orthogonal of its kernel, there 
exists Cl > such that 



\\x*-Xo\\2 ^ C^-'IMX* ~ Xo)\\2 



^^^k^W^k-PlMLljix* - Xo)\\2 



Noticing that 

\\LUx* - xo)\\2 = \\L*s,Vi^^Li,^^{x* - a;o)||2, 
we apply Lemma |2] to get 

||a^*-a^o||2 ^ C^'^WHx* - xo)\\2 

Using Lemma [T] yields the desired result. 



E. Proof of Corollary \3\ 

Take a = cq + Leo + us„ + {Lso)~^^*z. First, axo = eo 
since eo e Tq and Im(r) C Im{{Ls„)+) = lm{L*gJ. Then 
I^Soll^A = ICa,j(To,eo) < 1, whence we get that a e d\\ ■ 
\\a{L*xo). 

Now, we observe by definition of S that 7'kcr(L* )($*$5 — 
Id)LT(, = 0, which implies that Im(($*$S - Id)LTo)) C 
Im(Ls„). In turn, Ls„T = is„(iSo)+ (($**2 - Id)LTj = 



Im(Lso 



(($*$S-Id)LTo) = ($*$S - Id)LTo. This, to- 



gether with the fact that u e kcr(Lso) and $*z g Im(Lso) 
yields 

Ls^a = ($*$S - Id)LToeo + **z 

= $*77 — LTgCt <F=^ <1>*77 = La , 

which implies that <1>*77 = La £ dR{xo). We have just shown 
that the vectors a and 77 as given above satisfy the source 
condition (SC^jo) and the dual non-saturation condition. We 
conclude by applying Theorem |2] using (INJtq). ■ 

V. Conclusion 

We provided a unified analysis of the structural properties of 
regularized solutions to linear inverse problems through a class 
of semi-norms formed by composing decomposable norms 
with a linear operator We provided conditions that guarantee 
uniqueness, and also those ensuring stability to bounded noise. 
The stability bound was achieved without requiring (even 
partial) recovery of Tq and eo. Recovery of Tq and eo for 
analysis-type decomposable priors and beyond is currently 
under investigation. Another perspective concerns whether the 
£2 bound on x* — xo can be extended to cover more general 
low complexity-inducing regularizers beyond decomposable 



norms. 
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